Introduction¶
A musical piece often consists of recurring elements at various levels, from motifs to phrases to sections such as verse-chorus. To generate a coherent piece, a model needs to reference elements that came before, sometimes in the distant past, repeating, varying, and further developing them to create contrast and surprise. But before we delve into the technical implementation let us understand the building blocks of music:
Labels of the notes are (in sharp, #, notation):
C# D# F# G# A#
C D E F G A B ...
Labels of the notes are (in flat, $\flat$, notation):
Db Eb Gb Ab Bb
C D E F G A B ...
The A in the 4th octave is typically tuned at 440 Hz
- A half step is the smallest increment you can make
- After twelve half-steps you're back to the same note, but it sounds exactly twice as high
- In standard piano tuning, the frequency is multiplied by ${}^{12}\sqrt{2} \approx 1.059$
Frequency of note is implemented as:
$$f = f_{\mathrm{A4}}\bigg( {}^{12}\sqrt{2} \bigg)^ N $$where $N$ is the number of steps needed (can be negative) to move from A4 to the desired note.
Scale: A scale is a selection of notes that fit well together.
Chords: A chord is any harmonic set of pitches/frequencies consisting of multiple notes that are heard as if sounding simultaneously.
Arpeggio: An arpeggio is a type of broken chord in which the notes that compose a chord are individually sounded in a progressive rising or descending order.
Reference:¶
- https://www.youtube.com/watch?v=hXrpV2ffJRU&ab_channel=JustinGuitar
- https://github.com/marcelraas/music-generator/blob/master/presentation/2-music-generation.ipynb
Now that we know the building blocks of music, let us understand how we can generate music with deep learning. We take a language-modeling approach to training generative models for symbolic music. Hence we represent music as a sequence of discrete tokens, with the vocabulary determined by the dataset.
The JSB Chorale dataset consists of four-part scored choral music, which can be represented as a matrix where rows correspond to chords and columns to time discretized notes. The matrix’s entries are integers that denote which pitch is being played. Notes range from 36 (C1 = C on octave 1) to 81 (A5 = A on octave 5), plus 0 for silence:
This is very similar to time-series data or word sequence data in NLP. So we took a sequence to sequence modeling approach for generating the output note sequences. Each chorale will be a long sequence of notes (rather than chords), and we can just train a model that can predict the next note given all the previous notes. We will feed a window to the neural net, and it tries to predict that same window shifted one time step into the future.
